Electronic health records (EHRs) in clinical research and platform trials: Application of the innovative EHR-based methods developed by EU-PEARL.

OBJECTIVE: Electronic Health Record (EHR) systems are digital platforms in clinical practice used to collect patients' clinical information related to their health status and represents a useful storage of real-world data. EHRs have a potential role in research studies, in particular, in platform trials. Platform trials are innovative trial designs including multiple trial arms (conducted simultaneously and/or sequentially) on different treatments under a single master protocol. However, the use of EHRs in research comes with important challenges such as incompleteness of records and the need to translate trial eligibility criteria into interoperable queries. In this paper, we aim to review and to describe our proposed innovative methods to tackle some of the most important challenges identified. This work is part of the Innovative Medicines Initiative (IMI) EU Patient-cEntric clinicAl tRial pLatforms (EU-PEARL) project's work package 3 (WP3), whose objective is to deliver tools and guidance for EHR-based protocol feasibility assessment, clinical site selection, and patient pre-screening in platform trials, investing in the building of a data-driven clinical network framework that can execute these complex innovative designs for which feasibility assessments are critically important. METHODS: ISO standards and relevant references informed a readiness survey, producing 354 criteria with corresponding questions selected and harmonised through a 7-round scoring process (0-1) in stakeholder meetings, with 85% of consensus being the threshold of acceptance for a criterium/question. ATLAS cohort definition and Cohort Diagnostics were mainly used to create the trial feasibility eligibility (I/E) criteria as executable interoperable queries. RESULTS: The WP3/EU-PEARL group developed a readiness survey (eSurvey) for an efficient selection of clinical sites with suitable EHRs, consisting of yes-or-no questions, and a set-up of interoperable proxy queries using physicians' defined trial criteria. Both actions facilitate recruiting trial participants and alignment between study costs/timelines and data-driven recruitment potential. CONCLUSION: The eSurvey will help create an archive of clinical sites with mature EHR systems suitable to participate in clinical trials/platform trials, and the interoperable proxy queries of trial eligibility criteria will help identify the number of potential participants. Ultimately, these tools will contribute to the production of EHR-based protocol design.

TransFAIR study: a European multicentre experimental comparison of EHR2EDC technology to the usual manual method for eCRF data collection.

PURPOSE: Regulatory authorities including the Food and Drug Administration and the European Medicines Agency are encouraging to conduct clinical trials using routinely collected data. The aim of the TransFAIR experimental comparison was to evaluate, within real-life conditions, the ability of the Electronic Health Records to Electronic Data Capture (EHR2EDC) module to accurately transfer from EHRs to EDC systems patients' data of clinical studies in various therapeutic areas. METHODS: A prospective study including six clinical trials from three different sponsors running in three hospitals across Europe has been conducted. The same data from the six studies were collected using both traditional manual data entry and the EHR2EDC module. The outcome variable was the percentage of data accurately transferred using the EHR2EDC technology. This percentage was calculated considering all collected data and the data in four domains: demographics (DM), vital signs (VS), laboratories (LB) and concomitant medications (CM). RESULTS: Overall, 6143 data points (39.6% of the data in the scope of the TransFAIR study and 16.9% when considering all data) were accurately transferred using the platform. LB data represented 65.4% of the data transferred; VS data, 30.8%; DM data, 0.7% and CM data, 3.1%. CONCLUSIONS: The objective of accurately transferring at least 15% of the manually entered trial datapoints using the EHR2EDC module was achieved. Collaboration and codesign by hospitals, industry, technology company, supported by the Institute of Innovation through Health Data was a success factor in accomplishing these results. Further work should focus on the harmonisation of data standards and improved interoperability to extend the scope of transferable EHR data.

How to Improve Cancer Patients ENrollment in Clinical Trials From rEal-Life Databases Using the Observational Medical Outcomes Partnership Oncology Extension: Results of the PENELOPE Initiative in Urologic Cancers.

PURPOSE: To compare the computability of Observational Medical Outcomes Partnership (OMOP)-based queries related to prescreening of patients using two versions of the OMOP common data model (CDM; v5.3 and v5.4) and to assess the performance of the Greater Paris University Hospital (APHP) prescreening tool. MATERIALS AND METHODS: We identified the prescreening information items being relevant for prescreening of patients with cancer. We randomly selected 15 academic and industry-sponsored urology phase I-IV clinical trials (CTs) launched at APHP between 2016 and 2021. The computability of the related prescreening criteria (PC) was defined by their translation rate in OMOP-compliant queries and by their execution rate on the APHP clinical data warehouse (CDW) containing data of 205,977 patients with cancer. The overall performance of the prescreening tool was assessed by the rate of true- and false-positive cases of three randomly selected CTs. RESULTS: We defined a list of 15 minimal information items being relevant for patients' prescreening. We identified 83 PC of the 534 eligibility criteria from the 15 CTs. We translated 33 and 62 PC in queries on the basis of OMOP CDM v5.3 and v5.4, respectively (translation rates of 40% and 75%, respectively). Of the 33 PC translated in the v5.3 of the OMOP CDM, 19 could be executed on the APHP CDW (execution rate of 58%). Of 83 PC, the computability rate on the APHP CDW reached 23%. On the basis of three CTs, we identified 17, 32, and 63 patients as being potentially eligible for inclusion in those CTs, resulting in positive predictive values of 53%, 41%, and 21%, respectively. CONCLUSION: We showed that PC could be formalized according to the OMOP CDM and that the oncology extension increased their translation rate through better representation of cancer natural history.

AP-HP Health Data Space (AHDS) to the Test of the Covid-19 Pandemic.

Sharing observational and interventional health data within a common data space enables university hospitals to leverage such data for biomedical discovery and moving towards a learning health system. OBJECTIVE: To describe the AP-HP Health Data Space (AHDS) and the IT services supporting piloting, research, innovation and patient care. METHODS: Built on three pillars - governance and ethics, technology and valorization - the AHDS and its major component, the Clinical Data Warehouse (CDW) have been developed since 2015. RESULTS: The AP-HP CDW has been made available at scale to AP-HP both healthcare professionals and public or private partners in January 2017. Supported by an institutional secured and high-performance cloud and an ecosystem of tools, mostly open source, the AHDS integrates a large amount of massive healthcare data collected during care and research activities. As of December 2021, the AHDS operates the electronic data capture for almost +840 clinical trials sponsored by AP-HP, the CDW is enabling the processing of health data from more than 11 million patients and generated +200 secondary data marts from IRB authorized research projects. During the Covid-19 pandemic, AHDS has had to evolve quickly to support administrative professionals and caregivers heavily involved in the reorganization of both patient care and biomedical research. CONCLUSION: The AP-HP Data Space is a key facilitator for data-driven evidence generation and making the health system more efficient and personalized.

Reliability of Drug-Drug Interaction Measurement on Real-Word Data: The ReMIAMes Project.

The ReMIAMes project proposes a methodological framework to provide a reliable and reproducible measurement of the frequency of drug-drug interactions (DDI) when performed on real-world data. This framework relies on (i) a fine-grained and contextualized definition of DDIs, (ii) a shared minimum information model to select the appropriate data for the correct interpretation of potential DDIs, (iii) an ontology-based inference module able to handle missing data to classify prescription lines with potential DDIs, (iv) a report generator giving the value of the measurement and explanations when potential false positive are detected due to a lack of available data. All the tools developed are intended to be publicly shared under open license.

Performance evaluation of three semantic expansions to query PubMed.

BACKGROUND: PubMed is one of the most important basic tools to access medical literature. Semantic query expansion using synonyms can improve retrieval efficacy. OBJECTIVE: The objective was to evaluate the performance of three semantic query expansion strategies. METHODS: Queries were built for forty MeSH descriptors using three semantic expansion strategies (MeSH synonyms, UMLS mappings, and mappings created by the CISMeF team), then sent to PubMed. To evaluate expansion performances for each query, the first twenty citations were selected, and their relevance were judged by three independent evaluators based on the title and abstract. RESULTS: Queries built with the UMLS expansion provided new citations with a slightly higher mean precision (74.19%) than with the CISMeF expansion (70.28%), although the difference was not significant. Inter-rater agreement was 0.28. Results varied greatly depending on the descriptor selected. DISCUSSION: The number of citations retrieved by the three strategies and their precision varied greatly according to the descriptor. This heterogeneity could be explained by the quality of the synonyms. Optimal use of these different expansions would be through various combinations of UMLS and CISMeF intersections or unions. CONCLUSION: Information retrieval tools should propose different semantic expansions depending on the descriptor and the search objectives.

Performances of a Solution to Semi-Automatically Fill eCRF with Data from the Electronic Health Record: Protocol for a Prospective Individual Participant Data Meta-Analysis.

Clinical trial data collection still relies on a manual entry from information available in the medical record. This process introduces delay and error risk. Automating data transfer from Electronic Health Record (EHR) to Electronic Data Capture (EDC) system, under investigators' supervision, would gracefully solve these issues. The present paper describes the design of the evaluation of a technology allowing EHR to act as eSource for clinical trials. As part of the EHR2EDC project, for 6 ongoing clinical trials, running at 3 hospitals, a parallel semi-automated data collection using such technology will be conducted focusing on a limited scope of data (demographic data, local laboratory results, concomitant medication and vital signs). The evaluation protocol consists in an individual participant data prospective meta-analysis comparing regular clinical trial data collection to the semi-automated one. The main outcome is the proportion of data correctly entered. Data quality and associated workload for hospital staff will be compared as secondary outcomes. Results should be available in 2020.

Identification of the Best Semantic Expansion to Query PubMed Through Automatic Performance Assessment of Four Search Strategies on All Medical Subject Heading Descriptors: Comparative Study.

BACKGROUND: With the continuous expansion of available biomedical data, efficient and effective information retrieval has become of utmost importance. Semantic expansion of queries using synonyms may improve information retrieval. OBJECTIVE: The aim of this study was to automatically construct and evaluate expanded PubMed queries of the form "preferred term"[MH] OR "preferred term"[TIAB] OR "synonym 1"[TIAB] OR "synonym 2"[TIAB] OR …, for each of the 28,313 Medical Subject Heading (MeSH) descriptors, by using different semantic expansion strategies. We sought to propose an innovative method that could automatically evaluate these strategies, based on the three main metrics used in information science (precision, recall, and F-measure). METHODS: Three semantic expansion strategies were assessed. They differed by the synonyms used to build the queries as follows: MeSH synonyms, Unified Medical Language System (UMLS) mappings, and custom mappings (Catalogue et Index des Sites Médicaux de langue Française [CISMeF]). The precision, recall, and F-measure metrics were automatically computed for the three strategies and for the standard automatic term mapping (ATM) of PubMed. The method to automatically compute the metrics involved computing the number of all relevant citations (A), using National Library of Medicine indexing as the gold standard ("preferred term"[MH]), the number of citations retrieved by the added terms ("synonym 1"[TIAB] OR "synonym 2"[TIAB] OR …) (B), and the number of relevant citations retrieved by the added terms (combining the previous two queries with an "AND" operator) (C). It was possible to programmatically compute the metrics for each strategy using each of the 28,313 MeSH descriptors as a "preferred term," corresponding to 239,724 different queries built and sent to the PubMed application program interface. The four search strategies were ranked and compared for each metric. RESULTS: ATM had the worst performance for all three metrics among the four strategies. The MeSH strategy had the best mean precision (51%, SD 23%). The UMLS strategy had the best recall and F-measure (41%, SD 31% and 36%, SD 24%, respectively). CISMeF had the second best recall and F-measure (40%, SD 31% and 35%, SD 24%, respectively). However, considering a cutoff of 5%, CISMeF had better precision than UMLS for 1180 descriptors, better recall for 793 descriptors, and better F-measure for 678 descriptors. CONCLUSIONS: This study highlights the importance of using semantic expansion strategies to improve information retrieval. However, the performances of a given strategy, relatively to another, varied greatly depending on the MeSH descriptor. These results confirm there is no ideal search strategy for all descriptors. Different semantic expansions should be used depending on the descriptor and the user's objectives. Thus, we developed an interface that allows users to input a descriptor and then proposes the best semantic expansion to maximize the three main metrics (precision, recall, and F-measure).

Initializing a hospital-wide data quality program. The AP-HP experience.

BACKGROUND AND OBJECTIVES: Data Quality (DQ) programs are recognized as a critical aspect of new-generation research platforms using electronic health record (EHR) data for building Learning Healthcare Systems. The AP-HP Clinical Data Repository aggregates EHR data from 37 hospitals to enable large-scale research and secondary data analysis. This paper describes the DQ program currently in place at AP-HP and the lessons learned from two DQ campaigns initiated in 2017. MATERIALS AND METHODS: As part of the AP-HP DQ program, two domains - patient identification (PI) and healthcare services (HS) - were selected for conducting DQ campaigns consisting of 5 phases: defining the scope, measuring, analyzing, improving and controlling DQ. Semi-automated DQ profiling was conducted in two data sets - the PI data set containing 8.8 M patients and the HS data set containing 13,099 consultation agendas and 2122 care units. Seventeen DQ measures were defined and DQ issues were classified using a unified DQ reporting framework. For each domain, actions plans were defined for improving and monitoring prioritized DQ issues. RESULTS: Eleven identified DQ issues (8 for the PI data set and 3 for the HS data set) were categorized into completeness (n = 6), conformance (n = 3) and plausibility (n = 2) DQ issues. DQ issues were caused by errors from data originators, ETL issues or limitations of the EHR data entry tool. The action plans included sixteen actions (9 for the PI domain and 7 for the HS domain). Though only partial implementation, the DQ campaigns already resulted in significant improvement of DQ measures. CONCLUSION: DQ assessments of hospital information systems are largely unpublished. The preliminary results of two DQ campaigns conducted at AP-HP illustrate the benefit of the engagement into a DQ program. The adoption of a unified DQ reporting framework enables the communication of DQ findings in a well-defined manner with a shared vocabulary. Dedicated tooling is needed to automate and extend the scope of the generic DQ program. Specific DQ checks will be additionally defined on a per-study basis to evaluate whether EHR data fits for specific uses.

Impact on medical practice of accessing pharmaceutical records.

INTRODUCTION: The pharmaceutical record system (PRS) is a French nationwide centralized electronic database shared among all community pharmacists listing all drugs dispensed by community pharmacists in the last four months. The objective of this study, the Medication Assessment Through Real time Information eXchange - Distributed Pharmaceutical Record System (MATRIX - DPRS) study, was to assess the clinical impact of the PRS upon granting access to physicians in three hospital specialties: anesthesiology, emergency medicine and geriatrics. MATERIAL AND METHODS: A multicenter prospective study was conducted in six hospital departments, two per specialty. Participating physicians noted medication information found exclusively in the pharmaceutical record (PR) of each patient unavailable elsewhere and any diagnostic or therapeutic management changes resulting from the PR information. The primary objective was to assess the proportion of diagnostic or therapeutic management changes attributable to the PR among patients who had an accessible PR. RESULTS: The inclusion level ranged from 1.1 to 30% in the six departments. The rate of diagnostic or therapeutic management changes was highest in geriatrics (n = 31/67; 46.3% 95% Confidence IntervaI (CI): 34.0-58.9%) and lowest in anesthesiology (n = 36/227; 15.9% 95% CI: 11.4-21.3%). Emergency medicine was intermediate (n = 5/22; 22.7% 95% CI: 7.8-45.4%). CONCLUSION: Although the inclusion rate and statistical precision were low, these findings suggest that the information contained in the PRS is useful and may result in modifying patient management in a sizeable proportion of patients. This opens the prospect of evaluating other hospital specialties, as well as primary and secondary care settings.

Terminology Coverage from Semantic Annotated Health Documents.

BACKGROUND: Unstructured health documents (e.g. discharge summaries) represent an important and unavoidable source of information. METHODS: A semantic annotator identified all the concepts present in the health documents from the clinical data warehouse of the Rouen University Hospital. RESULTS: 2,087,784,055 annotations were generated from a corpus of about 11.9 million documents with an average of 175 annotations per document. SNOMED CT, NCIt and MeSH were the top 3 terminologies that reported the most annotation. DISCUSSION: As expected, the most general terminologies with the most translated concepts were those with the most concepts identified.

Lost in translation? A multilingual Query Builder improves the quality of PubMed queries: a randomised controlled trial.

BACKGROUND: MEDLINE is the most widely used medical bibliographic database in the world. Most of its citations are in English and this can be an obstacle for some researchers to access the information the database contains. We created a multilingual query builder to facilitate access to the PubMed subset using a language other than English. The aim of our study was to assess the impact of this multilingual query builder on the quality of PubMed queries for non-native English speaking physicians and medical researchers. METHODS: A randomised controlled study was conducted among French speaking general practice residents. We designed a multi-lingual query builder to facilitate information retrieval, based on available MeSH translations and providing users with both an interface and a controlled vocabulary in their own language. Participating residents were randomly allocated either the French or the English version of the query builder. They were asked to translate 12 short medical questions into MeSH queries. The main outcome was the quality of the query. Two librarians blind to the arm independently evaluated each query, using a modified published classification that differentiated eight types of errors. RESULTS: Twenty residents used the French version of the query builder and 22 used the English version. 492 queries were analysed. There were significantly more perfect queries in the French group vs. the English group (respectively 37.9% vs. 17.9%; p < 0.01). It took significantly more time for the members of the English group than the members of the French group to build each query, respectively 194 sec vs. 128 sec; p < 0.01. CONCLUSIONS: This multi-lingual query builder is an effective tool to improve the quality of PubMed queries in particular for researchers whose first language is not English.

Querying EHRs with a Semantic and Entity-Oriented Query Language.

While the digitization of medical documents has greatly expanded during the past decade, health information retrieval has become a great challenge to address many issues in medical research. Information retrieval in electronic health records (EHR) should also reduce the difficult tasks of manual information retrieval from records in paper format or computer. The aim of this article was to present the features of a semantic search engine implemented in EHRs. A flexible, scalable and entity-oriented query language tool is proposed. The program is designed to retrieve and visualize data which can support any Conceptual Data Model. The search engine deals with structured and unstructured data, for a sole patient from a caregiver perspective, and for a number of patients (e.g. epidemiology). Several types of queries on a test database containing 2,000 anonymized patients EHRs (i.e. approximately 200,000 records) were tested. These queries were able to accurately treat symbolic, textual, numerical and chronological data.

Evaluation of the Terminology Coverage in the French Corpus LiSSa.

Extracting concepts from medical texts is a key to support many advanced applications in medical information retrieval. Entity recognition in French texts is moreover challenged by the availability of many resources originally developed for English texts. This paper proposes an evaluation of the terminology coverage in a corpus of 50,000 French articles extracted from the bibliographic database LiSSa. This corpus was automatically indexed with 32 health terminologies, published in French or translated. Then, the terminologies providing the best coverage of these documents were determined. The results show that major resources such as the NCI and SNOMED CT thesauri achieve the largest annotation of the corpus while specific French resources prove to be valuable assets.

Behavior and attitudes of residents and general practitioners in searching for health information: From intention to practice.

BACKGROUND: Physicians are increasingly encouraged to practice evidence-based medicine (EBM), and their decisions require evidence based on valid research. Existing literature shows a mismatch between general practitioners' (GPs) information needs and evidence available online. The aim of this study was to explore the attitudes and behavior of residents in general medicine and GPs when seeking medical information online. METHODS: Five focus groups (FGs) involving residents in general medicine and GPs were conducted between October 2013 and January 2014. The overall number of participants recruited was 35. The focus group discussion guide focused on participants' experiences in searching for health information on the Internet, perceived barriers and possible solutions for improving the quality of their own search processes. Descriptive analysis was performed by three researchers. RESULTS: Participants described a wide range of research topics, covering all general medicine core competencies, and especially patient-oriented topics. They used a limited list of websites. Participants were not confident about their ability to assess the quality of the information they found. Their assessment of data quality was based on intuition, and they mainly sought concordance with their existing knowledge. The way the data were exposed was considered very important. Participants were looking for information that was directly linked to their clinical practice. Information seeking processes varied among participants. They felt they had not mastered query building for conducting searches, and were aware of the impact this shortcoming had on the quality of their search for information. CONCLUSIONS: Residents in general medicine and GPs understood the importance of EBM and the need for objective and reliable information. The present study highlights the difficulties in identifying this kind of information, and suggests ideas for improvement. Available search tools should change in order to fill the gap with real-world clinical practice, for example by integrating a patient-centred approach.

Influence of author's affiliation and funding sources on the results of cohort studies on occupational cancer.

BACKGROUND: Reliability and credibility of research conducted by industry have been questioned, including in the field of occupational health. METHODS: Cohort studies on occupational cancer published between 2000 and 2010 were compared according to their results, their conclusions, their funding, and the affiliation of their authors. RESULTS: Overall, 510 articles were included. Studies published by authors with public affiliation or funded by public grants concluded that their study showed an excess of cancer more frequently (P = 0.01) than studies published by authors with private affiliation or funded by private grants (88% [95%CI = 85-91] vs. 73% [95%CI = 56-88] and 92% [95%CI = 86-97] vs. 71% [95%CI = 57-84], respectively). Discrepancies between statistical results and conclusion occurred more frequently in articles written by authors from the private sector than from the public sector (42% [IC95% = 26-60] vs. 23% [IC95% = 18-26], P = 0.02). CONCLUSIONS: Industry affiliations of authors or industry support of studies are associated with the results of published studies on occupational cancer. The underlying mechanisms warrant further investigation.

Quality indicators from laboratory and radiology information systems.

Consequences of the computerization of laboratory and radiology information system (LIS and RIS) are not well documented. The aim of this study was to evaluate the impact of computerization of LIS and RIS of four hospitals on performance and quality of care. The study was divided into three phases. First, the subprocesses and information flows of LIS and RIS were described. Then, a literature review was performed in order to identify the indicators used to assess the impact of computerization. Finally, comparisons were made between 2 hospitals. Using the initial framework, each partner described its process mapping concerning LIS and RIS. The review identified a wide panel of indicators. Only 41 were useful to assess the impact of information systems. For each two by two comparison, lists of relevant indicators have been selected from the identified indicators and according to the process mapping comparison. Two by two comparisons have to be completed. Eventually, these indicators may be integrated in the quality process of hospital information systems.

A search engine to access PubMed monolingual subsets: proof of concept and evaluation in French.

BACKGROUND: PubMed contains numerous articles in languages other than English. However, existing solutions to access these articles in the language in which they were written remain unconvincing. OBJECTIVE: The aim of this study was to propose a practical search engine, called Multilingual PubMed, which will permit access to a PubMed subset in 1 language and to evaluate the precision and coverage for the French version (Multilingual PubMed-French). METHODS: To create this tool, translations of MeSH were enriched (eg, adding synonyms and translations in French) and integrated into a terminology portal. PubMed subsets in several European languages were also added to our database using a dedicated parser. The response time for the generic semantic search engine was evaluated for simple queries. BabelMeSH, Multilingual PubMed-French, and 3 different PubMed strategies were compared by searching for literature in French. Precision and coverage were measured for 20 randomly selected queries. The results were evaluated as relevant to title and abstract, the evaluator being blind to search strategy. RESULTS: More than 650,000 PubMed citations in French were integrated into the Multilingual PubMed-French information system. The response times were all below the threshold defined for usability (2 seconds). Two search strategies (Multilingual PubMed-French and 1 PubMed strategy) showed high precision (0.93 and 0.97, respectively), but coverage was 4 times higher for Multilingual PubMed-French. CONCLUSIONS: It is now possible to freely access biomedical literature using a practical search tool in French. This tool will be of particular interest for health professionals and other end users who do not read or query sufficiently in English. The information system is theoretically well suited to expand the approach to other European languages, such as German, Spanish, Norwegian, and Portuguese.